The present invention has as its principal objective the control of a computer caused by stimuli in the body and brain. The present invention is thus primarily concerned with the mode of finding the radiating properties of the human brain and selectively applying these findings to the control of computerized devices. Additionally, stimuli of other places of the body are included in these teachings.
It has been known for nearly 30 years that the electrical activity of nerves and muscle cells create electric energy and magnetic fields that can be measured. To draw conclusions about the electric functions of organs from those fields, however, one has to know the spatial and temporal distribution of the biomagnetic fields. In 1988, the first biomagnetic multichannel system was introduced for clinical research enabling the acquisition of complete magnetic field patterns, noninvasively and in real time. The latest technology using superconductive gradiometers and SQUIDs (Superconducting Quantum Interference Device) were employed in this device. “The feasibility for noninvasively analyzing the cardiac excitation path, especially in localizing the accessory bundle in Wolff Parkinson White syndrome and regions causing extrasystoles and tachycardia, had already been demonstrated. An example of localization of epileptic foci in the interictal state, or slow wave activity in the brain.” Ref. R. Helle and A. Oppelt, FIRST EXPERIENCES FROM CLINICAL INSTALLATIONS OF BIOMAGNETIC MULTICHANNEL SYSTEMS, Siemens Medical Engineering Group, Henkestr. 127, D-W-8520, Erlangen, Germany.
The activity of biological cells, e.g. nerves, muscle fibers and brain matter, is electric in origin. In a physical context, excited cells can be considered as galvanic elements situated in a conductive medium—the body. Sometimes bundles of cells are active simultaneously. The activity of these cells can be modeled by an equivalent current dipole (i.e., a small battery), consisting of a current source and sink, separated by a short distance.
A current dipole sends a known volume of current into the conductive surrounding. When this current reaches the surface of the body, electric potentials can be measured with electrodes. Medical diagnosis measurements in the heart are shown on an electrocardiogram (ECG). Those in the head are shown on an electroencephalogram (EEG). ECGs and EEGs provide information on the time course of the current sources in the body. However, sufficient localization accuracy can not be attained with these methods due to the strong influence of local tissue conductivity, which can vary considerably.
The strength of the electric fields being measured generally depends on the strength of the source and on the position of the electrodes with respect to the source. Since the conductivity of different tissues normally is not known, localization of a current dipole in the body is only possible with electrodes if they are brought very close to the site of the electric activity. For example, to determine the origin of electric activity in the human heart, one has to work with catheters—an invasive procedure. A goal of this invention is to primarily provide thought control with noninvasive techniques. Likewise for the human brain, invasive procedures here create a much greater risk.
Every electric current is surrounded by a magnetic field which is, in essence, unaffected by the electromagnetic properties of tissue. Research into the localization of current dipoles using magnetic field measurements was performed as early as 1963. Ref. W. Moshage, S. Achenbach, S. Schneider, K. Gohl, K. Abraham-Fuchs, R. Graumann, and K. Bachmann, APPLICATION OF MULTICHANNEL SYSTEMS IN MAGNETOCARDIOGRAPHY, 1992 Elsevier Science Publishers B.V., Biomagnetism: Clinical Aspects. M. Hoke, Editor. The magnetic field, generated by current dipole in the human body and measured outside of the body, has two fields of activity: one originates from the current dipole itself and the other originates from the volume currents. While the influence of the first field of activity can easily be calculated from the Biot-Savart law, the second depends on the paths of the currents in the body.
In order to localize the current dipole, it is necessary to know the distribution of the magnetic field being measured. Generally, one measures a north pole where the magnetic field lines leave the human body and a south pole where they enter again. The current dipole lies in the center, between the two poles. The depth is determined by the distance between the two poles: the further the two poles are apart, the deeper the dipole is situated.
It is known that there are areas of the brain that emit magnetic energy in response to the thinking process. We know from ECG's and EEG's, monitors can record body changes. Further, Biofeedback studies and empirical data show some of the functional control that is possible with the brain and body.
Also known is Magnetic Source Imaging (MSI) equipment in use today, such as the “Krenicon” and the “Magnes,” which noninvasively monitor and record brain activity. This has many uses, including use as a diagnostic tool for brain disorders such as epilepsy. Today's MSI are multichanneled and include high pass filters, anti-aliasing filters and 14 bit A/Ds with generally a 4 kHz sampling rate. Siemens' (Germany) Krenicon has 127 output channels which can be directed to 127 areas of the brain. It can show medical imaging and a journey into the brain to cure a patient with epilepsy. The journey depended on Siemens' Krenicon, cooled by liquid helium, to detect magnetic signals one billion times smaller than the earth magnetic field.
In the U.S., Biomagnetic Technologies, Inc.'s Magnes offers 147 output channels. Ref. W. Moshage, S. Achenbach, A. Weikl, K. Gohl, K. Abraham-Fuchs, S. Schneider and K. Bachmann PROGRESS IN BIOMAGNETIC IMAGING OF HEART ARRHYTHMIAS Frontiers in European Radiology, Vol. 8, Eds Baert/Heuck, Springer-Verlag, Berlin, Heidelberg 1991.
The advantage of MSI is that it is noninvasive and that this invention may require stimulus from parts of the brain that may be quite a distance from the surface and at a variety of locations, e.g., electrodes may not be practical by themselves. The mobility of this “Thought Controlled System” (TCS), FIG. 1 will become greater as the technology advances and as TCS is mass produced.
There are difficulties encountered when using today's MSI technology. However, they are being overcome by numerous organizations throughout the world. As more and more MSI equipment is placed in service and as the race for economic feasibility and technical advancements are pursued, this invention becomes more attractive. Publications say the race is on among many companies to bring MSI technology to a stage where it is miniaturized, inexpensive and insensitive to interference. Furthermore, the main features may be proven by sensing with less expensive techniques until MSI becomes cost effective. Each advance is contributing toward a more economically feasible TCS as presented here. A few examples of efforts where challenges are being met are:
Two Krenicons are installed in hospitals for clinical research as well as for establishing the clinical relevance of biomagnetic diagnosis. In the Biomagnetic Center of the University of Erlangen (Germany), biomagnetic investigations are performed regularly in the field of cardiology, epilepsy and other neurologic disorders, such as transitory ischemic attacks and stroke. At the Karolinska Hospital, Stockholm, epilepsy is diagnosed and new methods for treating biomagnetically localized foci with high energy radiation are explored. The investigation of ventricular late fields started 1993. See earlier Ref. to R. Helle and A. Oppelt. Another challenge with current MSI equipment found in the same Ref. (Pg. 2, Col. 1) is: “Selecting a site requires special attention in order to keep the influence of moving iron masses such as cars and elevators to an acceptable minimum. Installation includes a special shock proof concrete foundation for a shielded room and an active shielding loop. Large consumers of electric energy like a subway can cause considerable interference, even over large distances, especially if the current varies slowly as the shielding effect of the chamber is lowest in the range below about 0.1 Hz.”
Spatial resolution of the biomagnetic localization is dependant on identifying and compensating. “ . . . sources of localization errors (may be) . . . coordinate transfer into the magnetic resonance (MR) image, system noise, ‘biologic’ noise from electrical background activity of the human body, and modeling inaccuracies. Influence of system noise with phantoms, and an error of 1-2 mm was found. The reproducibility of the head position in the MEG device and the MR imaging system is typically 2 mm for a point in the temporal region and about 4 mm for a point in the occipital region” Ref. Siefried Schneider. Ph. D., et al MULTICHANNEL BIOMAGNETIC SYSTEM FOR STUDY OF ELECTRICAL ACTIVITY IN THE BRAIN AND HEART, Radiology 1990; 176-825-830. This is being overcome with technology and will be further reduced with the use of a “helmet” system mentioned later in this application.
As TCS costs are reduced and become used more by one individual, the site locations will be less often in hospitals and large metropolitan areas but rather suburban and rural residential areas. Also, additional noise reduction techniques are being developed using signal processing and signal enhancing as mentioned later.
Signal processing is used for enhancing the evaluation of data amid noisy environments and to correct physiological interference. Signal preprocessing is performed on the biomagnetic data prior to source localization. For example, the signal preprocessing may include algorithms for noise suppression or separation of signals from different sources.
The end result of data evaluation in biomagnetic imaging is the reconstruction of bioelectric activity from the measured field distribution in time and space. Such a procedure may consist of several signal-processing steps: a) correction of each measurement channel, b) averaging of several cycles to improve signal-to-noise ratio (SNR), c) definition of a physiological model, d) reconstruction of the source of interference for use in noise and echo cancellations, e) automated or semi-automated validation of the reconstruction result, and f) viewing and comparing reconstructed three dimensional localization with other imaging methods.
Biomagnetic image reconstruction is critically dependent on signal fidelity. Signal distortions, such as dc offset and low frequency (i.e., below 0.1 Hz) noise, have to be removed without imposing new distortions. Dc offset and low frequency noise stem mainly from electronic noise in high-gain amplifiers, thermal magnetic noise in surrounding materials, movement of the torso, and mechanical vibrations. In some cases, the signal of interest is also influenced by a preceding physiological activity from organs, such as the user's heart. Special correction algorithms are used for these cases and, depending on the type of interference, different baseline correction techniques are applied.
The word “artifacts” is used in the field of body measurement sciences to refer to additional signals (usually interfering) resulting from the functions of body parts other than the signal of interest. Webster defines it “a characteristic product of human activity.” The Magnetoencephalograph (MEG) can pick up magnetic signals produced by the heart with a signal strength which may be equal to or even greater than the neuromagnetic signals of interest. These undesirable signals, called artifacts, in MEG data may contaminate the signal. Sporadic events may not be recognized or classified correctly and the error in source localization can increase. In this instance a correction algorithm is utilized to suppress artifacts produced by the heart. Ref. K. Abraham-Fuchs et al., IMPROVEMENT OF NEUROMAGNETIC LOCALIZATION BY MCG ARTIFACT CORRECTION IN MEG RECORDINGS, Pg. 1787, 1992, Elsevier Science Publishers B.V. An example of a heart artifact removal algorithm is to make use of the fact that the user's heart is electrically inactive during a portion or portions of the heart cycle. A time window of the preceding cycle is defined and the mean of the signal during the appropriate portion or portions is subtracted from each stimulus channel individually. This theory is supported on Pg. 6 of earlier Ref. to W. Moshage, “PROGRESS IN BIOMAGNETIC . . . ” 1991. Care must be taken however, (such) “algorithms usually do not achieve their purpose ideally, but leave a residual of distortion in the data, or even may introduce new distortions.” Ref. Klaus Abraham-Fuchs et al., EFFECT OF BIOMAGNETIC SIGNAL PROCESSING ON SOURCE LOCALIZATION ESTIMATED BY MEANS OF STATISTICAL SOURCE DISTRIBUTIONS 0-7803-0785-2/92$03.00 IEEE (Pg. 1768 Col. 1, Abstract).
The state of the art of improving signal recognition and identifying the location (Localizing) of areas, specific parts and especially specific thoughts is such that localizing can be performed in varying degrees of accuracy. Companies throughout the world recognize the need for improvement and are in a race to improve the state of the art. The utility, efficiency and attraction of thought control will constantly increase with advances in signal recognition.
Having said all this about improving the biomagnetic image reconstruction to a point of fine precision, much of the sophistication and expensive components may be eliminated when the TCS is applied to specific tasks. These low cost systems are arrived at with experimentation by the user combined with utilizing empirical history data. This occurs because of the unique characteristics exhibited as unexpected thought patterns are identified for each individual and chosen for control. Image reconstruction is an attractive and often necessary tool for experiments and configuring systems. However, after the system is configured and the user is completely checked out, the image reconstruction feature can be eliminated. This is useful when economy is paramount.
TCS offers a valuable contribution to humanity in that of offering paralysed individuals a means for controlling computerized devices without the need for physical movement. Of particular benefit would be a case where, even with total physical paralysis, a person could do a full day's work at a computer; or communicate; or socialize; etc.; etc.
Although it will be appreciated that the present invention is not limited to such applications, examples of applications in which the present invention is particularly useful are as follows:
TCS could be of assistance to persons who cannot use their hands, arms or any other part of their body to perform work or otherwise perform computer control. This invention would allow a quadriplegic to put in a full days work including but not limited to computer control of production line devices, doing visual sorting and other tasks.
This thought controlled system (TCS) will enable a paralyzed person to control automation in the home, control the movement of an automated wheelchair and control other assistance devices.
The speed alone may be sufficient justification for employing a TCS. A Ref. to substantiate the speed potential is, “With the possibility of localizing electrical sources (dipoles) with a time resolution of milliseconds, biomagnetic evaluation has the potential to become a valuable tool in functional diagnosis, especially in combination with anatomic imaging” See earlier (Pg. 6) Ref. to Siefried Schmeider. Ph.D.
The thought controlled system will be of assistance to a person who cannot speak. The TCS method used may be similar to typing into a computer with a keyboard where the text is converted to an audio output. In order to accomplish this with fewer key stroke equivalents, the user might be trained in the language of a court reporter's shorthand typewriter. Speeds are attained at least as fast as a person speaks. This, again, depends on the number of reliable stimuli that the MSI can reliably produce from the electrical impulses of the user's brain.
Using thought controlled systems, handicapped persons will continually uncover new and additional information concerning detectable stimulus. The handicapped have a great appetite for improving the quality of their life. It is envisioned that clubs and organizations will be formed of this elite group and that their accomplishments will be monitored by the medical profession for unprecedented scientific advancements. Improvements to society will be furthered as TCSs are shared during off hours and around the clock by people intent on contributing experimental time. All these incidents will proliferate through interest groups and user group organizations as technology advances to produce more affordable systems.
Other candidates for uses of a TCS may be the military, industrial production lines, the games industry, and researchers in: Biofeedback, Psychiatry, Metaphysics, Subconscious thought analysis, Source of specific thoughts, Patients' “self data finding”, Patient therapy, and Human interfaces for the phenomenal speed potential.
A reference for brain activity controlling a computer is a video game by Pope et al, U.S. Pat. No. 5,377,100. He teaches a method of encouraging attention by correlating video game difficulty with attention level. Pope's video game comprises a video display which depicts objects for interaction with a player and a difficulty adjuster which increases the difficulty level, e.g., action speed and/or evasiveness of the depicted object, in a predetermined manner. The electrical activity of the brain is measured at selected sites by an EEG. A Fast Fourier Transform (FFT) breaks down the generated wave to determine levels of awareness, e.g., activity in the beta, theta, and alpha frequency bands. A value is generated based on this measured electrical signal, which is indicative of the level of awareness. The difficulty level of the game is increased as the awareness level value decreases and vice versa.
Pope's computer games require sensing entire frequency bands, each as a stimulus, for control functions to play the game. Conversely, TCS teaches selection and utilization of one individual stimulus or more stimuli and considers the actual thoughts of the user. Many more forms of games and control systems are suggested with this application by utilizing virtually unlimited numbers and types of stimuli. Other forms of input to the computer can be performed with this method, such as described later. It can also be seen that entities or equipments other than computers can be controlled in a similar fashion, or that the computer can communicate with control peripheral equipment or other systems.
Another reference that is related to controlling a computer based on user physiology is U.S. Pat. No. 5,016,213 (Dilts et al.) which discloses a method and apparatus for controlling the position of an image on the screen of a computer using galvanic skin response (GSR), also known as Psycho galvanic reflex (PGR) or electrodermal reflex (EDR). In particular, the system teaches the introduction of a GSR amplifier circuit that couples to the game paddle port of a conventional computer, e.g., an Apple II computer. The GSR amplifier circuit is contained within a housing having GSR contacts that are located on the exterior of the housing for the user. When the user applies a finger to the GSR contacts, the GSR amplifier circuit utilizes the skin resistance available at the GSR contacts to create an electrical signal that changes in sense and amplitude directly with changes in the resistance sensed between the GSR electrodes. Furthermore, there is a product sold under the mark MINDRIVE™, by The Other 90% Technologies, Inc.™ of San Rafael, Calif. 94912-2669 which is believed to include a number of the features disclosed in U.S. Pat. No. 5,016,213 that is available for use with home computers. Among other things, MINDRIVE™ permits the user to operate a ski simulator, create art, a flight simulator, etc., on the computer using the GSR method. Conversely, TCS teaches the selection and utilization of one individual stimulus or more stimuli and considers the actual thoughts of the user.
Another feature to be found emanating from this TCS capability is personal identification from brain waves, not unlike the phenomenon of fingerprints, eye prints and voice prints.